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ABSTRACT 

The relative power of three possible experimental 
designs under the condition that data is to re analyzed by 
nonpar a metr ic techniques; the comparison of the power of each 
ncnpa came trie technique to its parametric analogue; and the 
comparison of relative powers using non pa rame tr ic and parametric 
techniques are discussed* The three nonparam etr ic techniques 
concerned are the Kruskal-Wallis test on data where experimental 
units have been randomly assigned to levels of the independent 
variable, Friedman's rank analysis of variance (ANOVA) on data in a 
randomized blocks design, and a nonparametr ic analysis of covariance 
(ANCOVA). The parametric counterparts are respectively; one way 
analysis of variance (ANOVA), two-way ANOVA, and parametric ANCOVA* 
Since? the non par a me trie tests are based on large sample 
approximations, the goodness of fit for small samples is also of 
concern. Statements of asymptotic relative efficiency and imprecision 
are reviewed as suggestive of small sample powers of the designs and 
statistics investigated* Results of a Monte Carlo investigation are 
provided to further illustrate small sample power and to provide data 
on goedness of fit. (Author/CK) 
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Consider the case of an experimenter who is interested in testing the 
Hypothesis that t levels of an independent variable have an identical effect 
on the dependent variable values of the units in a population. Given that 
he has unbiased estimates of treatment effects* the experimenter should choose 
for a fixed sample size a procedure that provides as statistically powerful a 
test as possible* The purposes of the present paper are to discuss the rela- 
tive power of three possible designs under the condition that data are to be 
analyzed by nonparametric techniques, to compare the power of each nonparamettic 
technique to the power of its parametric analogue, and to compare the relative 
powers using nonparametric techniques to the better known relative powers using 
parametric techniques. The three nonparametric techniques are the Krusk.il- 
Wallis test on data where experimental units have been simple random assigned 
to levels of the independent variable, Friedman 1 s rank ANOVA on data in a 
randomized blocks design* and ? nonparametric analysis of covariance (ANCOVA) 
which has recently been proposed by Quade (1967). The parametric analogues are 
respectively; one way analysis of variance (ANOVA), two way ANOVA* and para- 
metric ANCOVA, Since the nonparametric tests are based on large sample approxi- 
mations* the goodness of their fit for small samples is also of concern. State- 
ments of asymptotic relative efficiency and imprecision are reviewed as suggestive 
of the relative small sample powers of the designs and statistics investigated. 
Results of a Monte Carlo investigation are provided to speak more directly to the 
concern of small sample power and to provide data on goodness of fit. 

In a 1970 AEPA paper (Porter and McSweeney, 1970) we empirically investi- 
gated the small sample goodness of fit and power of the Kruskal-Wallis and Friedman 
Q tests, but did not provide empirical ni’s and powers for their parametric analogues. 
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Our 1970 AERA paper also investigated the effects of analyzing data in randomized 
block form with a Kruskal-Wallis test, and the effects of using Friedman's test 
on data wnere blocks were formed after initial random assignment. Here we extend 
last year's work by considering Quade's nonpar ame trie ANCOVA, comparing the 
small sample properties of nonparametric ANCOVA to our last year's data on 
Kruskal-Wallis and Friedman's tests, and by presenting empirical results for 
all three parametric analogues. Partial motivation for our study is provided 
by Quade (1967, p, 1198) when in reference to his nonparametric ANCOVA he states 
"Another disadvantage of the rank method at this time appears to be its unknown 
behavior for small samples, say with less than five to ten observations per 
group." Further motivation is provided by the apparent lack of information in 
the literature about the small sample power of different experimental designs 
for testing a common hypothesis when nonparametric tests are used. By com- 
parison considerable attention has been given to study of the relative small 
sample power among those same experimental designs when parametric te 3 ts are 
u3ed. In another IS 7 1 AERA paper (McSweeney and Porter, 1971) we further 
extend our considerations of small sample goodness of fit and power to include 
a modification of Quade's nonparametric rank ANCOVA and a nonparametric index 
of response which we propose. 

When an experimenter is interested in testing the above stated hypothesis 
that t levels of an independent variable have an identical effect on the depen- 
dent variable values of the units in a population, he can proceed by obtaining 
a simple random sample from the population of interest and then use simple ran- 
dom assignment or experimental units to levels of the independent variable. 

(liven the assumptions of normality, homoscedasticity and independence he can 
analyze his data by a one-way ANOVA. However, if antecedent information is 
available on a concomitant variable and if the concomitant variable has a 
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linear relationship with the dependent variable, ANCOVA will probably provide 
a more powerful test. The well known assumptions for parametric ANCOVA are 
conditional njrmality, conditional homoscedasticity , independence, and parallel 
treatment group regression lines. 

Another method of using antecedent information on a concomitant variable 
in an attempt to improve statistical power is to employ a randomized blocks 
design. Pingel (1969) defines two types of randomized blocks design and com- 
pares their precisions when data are analyzed by a two way ANOVA. Unfortunately, 
the more precise method involves forming blocks on the population distribution 
of the concomitant variable and thus requires more information than is typically 
available. The other type of randomized blocks design requires a random sample 
of tb units from the population, which are then put in rank order on the basis 
of their concomitant variable values. Then the experimenter randomly assigns 
one unit to each level of the independent variable from the first set of t 
units, one unit to each level from the second set of t units, and so on until 
the tb observations have all been assigned. The second type of randomized 
blocks design is the one investigated in the present study because of its 
greater feasibility for an experimenter. The assumptions for two way AN07A 
mi data in a randomized blocks design are normality, homoscedasticity , inde- 
pendence, and no block by treatment interaction. 

Over the past several years there has been some debate as to che general 
u tility of nonparame trie techniques. The robustness of the analysis of variance 
F test (Box, 1955) has been offered as an argument in favor of its use, even 
when the assumptions of normality and homoscedasticity are violated. This argu- 
ment emphasizes the goodness-of-f it of the F test statistic t-j the theoretical 
null distribution ever when the assumptions for the F test are seriously in error. 
Its principal concern is with the size of the actual Type I error. Despite the 
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good control over Type I error given by the F test when parametric assumptions 
are violated, nonparametric tests for the same designs are generally more power- 
ful than their parametric counterparts in these circumstances (Brad?^./, 1968). 
Therefore, the Kruskal -Wallis , Friedman’s ANOVA, and Quade's nonparametric ANCOVA 
are to be preferred over their parametric counterparts when parametric assumptions 
have been violated. 

Both the Kruskal-Walli8 and the Friedman test statistics operate on ranks, 
but they differ in their methods of ranking. When calculating a Kruskal-Wallls 
test statistic the units are ranked 1 through tb on *-he basis of their dependent 
variable values. The Friedman test statistic requires that the units within each 
block be ranked 1 through t on the basis of their dependent variable values. The 
Kruskal-Wallis test statistic is then defined 



H= 



12 



l Rj, - 3 (tb-hl) , 



tb^(tb+l) T**l 
and the Friedman test statistic is defined 



2 

X “ 



U 



t' tbTttT) 1*1 - 3Mt+1) ' 



where R^denotes the sum of the b ranks under the Tth level of the independent 
variable. The assumptions for the Kruskal-Wallis test are independence of units 
and measurement of the dependent variable on at least ordinal scale. The assump- 
tions for the Friedman test are independence among blocks and that within each 
block the dependent variable is measured on at least an ordinal scale. 

Because the nonparametric ANCOVA proposed by Quade is relatively new and 
therefore probably not well known, a more detailed description is appropriate. 
Although Quade's nonparametric ANCOVA is not restricted to use of a single con- 
comitant variable, the present paper considers only the single concomitant vari- 
able case. The restriction is made to facilitate comparison with our earlier 
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investigation of Friedman's ANOVA on data in a randomized blocks design where 
we considered only a single concomitant variable. The nonparametr ic analysis 
of covariance test statistic is calculated by first replacing each observation 
on the concomitant or covariable by its respective rank and each observation 
on the dependent variable by its respective rank, where In both cases ranking 
is done on all experimental units across all levels of the independent variable. 
Each rank on the covariable is then replaced by its deviation from the mean of 
ranks on the covariable, and each rank on the dependent variable is replaced by 
its deviation from the mean of the ranks on the dependent variable. Disregarding 
levels of the independent variable, least squares theory is used on the deviation 
data to provide predicted deviations on the dependent variable. Because both 
sets of deviation data have a mean of zero and common variance, the least squares 
regression equation is 

" y tg S ^TB * 



where r is the Spearman rank order correlation coefficient calculated on 
the original rank data ignoring levels of the independent 
variable, 

d denotes a deviation rank, 

X denotes the covariable, 

Y denotes the dependent variable, 

T denotes the Tth levil of the independent variable, 

and B denotes the Bth replication within the Tth level of the inder^ndent 
variable. 

A new variable, say Z^, is formed by subtracting each unit's predicted deviation 
from its observed deviation, 



TB 



TB 






TB 
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The new variable, Z , becomes the dependent variable for a one-way ANOVA F test. 

i b 

The reference distribution for the F test statistic is the central F with degrees 
of freedom equal to t - 1 and tb - t as tb -► 00 . 

The only assumption that Quade's nonparametric ANCOVA has in common with 
parametric ANCOVA is independence of units. The nonparametric ANCOVA further 
assumes that the covariable has the same distribution in each population. The 
identical covariable distributions assumption is necessarily met if observations 
on the covariable are taken prior to the experiment and if simple random assign- 
ment of units to levels of the independent variable is employed. The relaxation 
of the need for a linear relationship between the dependent variable and covariable 
to only a need for a monotonic relationship for nonparametric ANCOVA may be of 
particular practical importance. It is also helpful to note that Quade's nonpara- 
meiric ANCOVA reduces to a Kruskal-Wallis test when the Spearman rank order cor- 
relation is zero. 

The question of goodness of fit of the three nonparametric test statistic's 
sampling distributions to their large sample approximations must be answered em- 
pirically. Their relative powers and comparisons of their powers to the powers 
of their respective parametric analogues for small samples must also be answered 
largely on an empirical basis; however, statements of Pitman asymptotic relative 
efficiency are informative, Quade (1967) has shown that the asymptotic relative 

efficiency of his nonparametric ANCOVA with respect to the Kruskal-Wallis test 
2 

is l/(l-pg) where pg denotes the population Spearman correlation. Asymptotic 
relative efficiency statements for Friedman with respect to Kruskal-Wallis and 
nonparametric ANCOVA with respect to Friedman are not known. 

When comparing the efficiency of nonparametric tests to their parametric 
counterparts given the parametric assumptions, Andrews (1954) has shown that the 
"ruskal-Wallis test has asymptotic relative efficiency of ,955 with respect to 
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the one-way ANOVA. Friedman (1937) has given the asymptotic relative efficiency 

of his test relative to the two-way ANOVA as .637 when t equals 2 and .912 when 

b equals 2. For other values of b and t the Friedman test has asymptotic relative 

efficiency between .637 and .912 (Noether, 1967). When the number of blocks for 

the Friedman test becomes large, its asymptotic relative efficiency with respect 

to the two-way ANOVA is .955 l_t ) , Quade (1967) demonstrated that his non- 

tH x 

7 2 

parametric ANCOVA has .955 (1 - )/(l - P ) asymptotic relative efficiency 

with respect to the parametric ANCOVA, where denotes the population cor- 
relation of the dependent variable and the covariable and p is the population 
Spearman correlation. The relationship between the two coefficients is 
PxY = 2 sin (ttp s /6) , 

given that X and Y have a bivariate normal distribution. It follows that the 
asymptotic relative efficiency of nonparametric to parametric ANCOVA is greatest 
when “ 0 and becomes less as the absolute value of increases, having a 
smallest value of .866. 

Cox (1957) provides indices of true imprecision for two-way ANOVA of data 
in randomized block form and parametric ANCOVa, which are instructive in choosing 
among the two techniques on the basis of their statistical power. Cox defines an 
index of true imprecision as the ratio of the average variance of the difference 
of means for pairs of levels of the independent variables over the average variance 
of the difference of means for pairs of levels of the independent variable if all 
of the variance of the dependent variable accounted for by the concomitant variable 
were removed. The index of true imprecision of two-way ANOVA on data in the type 
of randomized b)ock form of interest is 



1 




W, 



where W is the expected mean square of X within blocks divided by the variance 
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of X. The index of true imprecision for parametric ANCOVA is 

tb - t - 1 
tb - t - 2 

Based upon the above two statements of imprecision Cox concludes that where 
P^Y is less than .6, the randomized blocks design is to be preferred over 
ANCOVA; but where Is equal to or greater than .8, ANCOVA is to be pre- 
ferred over randomized blocks. These statements do not reflect the differences 
in number of degrees of freedom for the two parametric tests. When taken into 
consideration the differences in degrees of freedom suggest that a one-way 
ANOVA will be more powerful than either ANCOVA or randomized blocks, and that 
ANCOVA will be more powerful than randomized blocks when p^y = .0. 

Although the above statements of asymptotic relative efficiency and true 
imprecision provide no direct information about the small sample relative 
powers of the technique being investigated, they are suggestive. In general 
they suggest that given parametric assumptions the nonparame trie techniques 
should be expected to yield slightly less power than their parametric ana- 
logues. Further, they suggest that as the absolute value of p w increases 
the power of the nonparame trie ANCOVA should become progressively greater 
than the power of the Kruskal-Wallis . By a more indirect argument they sug- 
gest that Friedman ANOVA on data in randomized block fom should be more 
powerful than the Kruskal-Wallis test for fairly large absolute values of 
P^Y* They are somewhat less suggestive of what to expect when comparing 
the small sample powers of Friedman’s test to those for nonparame trie ANCOVA. 

Monte Carlo studies were conducted to provide information about the 
small sample properties of the sampling distributions of the Kruskal-Wallts 
and one-way ANOVA test statistics when calculated on data in a simple ran- 
dom assignment design; the nonparametri'i ANCOVA and parametric ANCOVA test 
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statistics when calculated on aata in a simple random assignment design with infor- 
mation on an antecedent concomitant variable available; and Friedman's ANOVA 
end two-way ANOVA when calculated on data in a randomized blocks design. Em- 
pirical esticu "es of the probability of a Type I error (a) provide information 
about the email sample goodness of fit for each of the three nonparametric 
statistics. Empirical estimates of power provide data relevant to choosing 
among the three nonparametric statistics, the three parametric statistics, 
and are indicative of the difference in power of nonparametric and parametric 
statistics vrhen parametric assumptions are met* 

Figure 1 presents an overview of the conditions under which each type 
of sampling distribution was investigated. For each X in Figure 1, sampling 
distributions for 1000 samples were generated for the central case, i.e. 

Ho: true, and one noncentral case, i.e. Ha: true. The population distri- 

butions indicated in Figure 1 were chosen to allow comparisons with the 
results from our earlier study of the Kruskal-Wallis and Friedman tests. 

Three values of t were investigated because of possible trends in the sampling 
distributions as the number of levels of the treatment independent variable 
increase. The smallest value of t was three since the Wilcoxon matched pairs 
test offers a more powerful alternative to the Friedman when t equals two. 

Three values of b were investigated because of possible trends in the sampling 
distributions as the number of unit3 under each level of the independent vari- 
able increase, The smallest value of b was five since exact tests would be 
more appropriate for smaller values. Four values of the correlation between 
the concomitant variable and the dependent variable were investigated to pro- 
vide information on the relative small sample powers of three nonparametric 
statistics, and to see if the interrelationships of their powers parallel the 



hpt-ter known interrelationships of the powers of their parametric analogues. 
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Figure 1 



CONDITIONS OF THE DESIGNS INVESTIGATED 
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The smallest value of the correlation was zero, defining the situation where 
the three designs are essentially the same; and the largest value was .8 since 
in practice correlations are infrequently larger. The non-central case was 
created by adding the value one to the dependent variable value of each unit 
under the first level of the treatment independent variable. For the three 
nonparametric tests the noncentral case was created prior to forming ranks. 

The non-central case was chosen because it seemed to represent a deviation 
from the null hypothesis that most experimenters would wish to notice. Fur- 
ther, it produced intermediate values of power which facilitated comparisons 
of the various techniques. 

All data generation and subsequent calculations were done on a Control 
Data 3600 computer. A pseudo-random unit normal deviate was generated by 
first generating random numbers from a uniform distribution by the multi- 
plicative congruent method. By the Central Limit Theorem the sum of six- 
teen of the above numbers is approximately norrully distributed, and by a 
linear rescaling the sura becomes a pseudo-random unit normal deviate. In 
this way 1,000 random samples of size tb each were generated for each pop- 
ulation indicated in Figure 1. Each population had a bivariate normal dis- 
tribution with known correlation between the concomitant and dependent vari- 
ables, and each marginal distribution normal zero, one. Given a sample of 
size tb, the data were then arranged into simple random assignment and ran- 
domized blocks designs. 

Table 1 compares the empirical sampling distributions of the thr^e non- 
parametric tests when calculated on data in their respective appropriate designs 
for t - 3, b - 8, and varying values of p^. The data for Kruskal-Wallis *nd 
Friedman teats are taken lrom our 1970 AERA paper (Porter and McSweeney, 1970). 

O he three rows labeled central J ’icate the goodness of fit of the empirical 
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distributions to their respective Chi Square and F reference distributions under 
the null hypothesis. The standard errors of the proportions representing em- 
pirical a f s are approximately ,009 for a - ,10, ,007 for ot 3 ,05, and .003 for 
a = ,01, The empirical and nominal values of a are in close agreement for all 
three statistics with the greatest discrepancies at ot = ,10 for the Friedman, 

As noted in our 1970 paper the largest discrepancies for the Fr. jdman are to 
be expected since the empirical values are closer to the values from the exact 
distribution of the Friedman test derived from permutation theory than are the 
nominal values* The discrepancy between the exact and large sample approxi- 
mate i distributions is reflecting the fact that the Friedman test statistic 
is a discrete variable with fewer possible values than the Kruskal-Wallis 
or nonparametric ANCOVA tests because of the difference in their methods of 
ranking. 

The three rows labeled non-central contain the empirical values of 
power for the three nonparametric tests given that a common alternative hy- 
pothesis was true. Their largest standard error of approximately .016 occurs 
when power equals ,5, As noted in our 1970 paper, the data clearly indicate 
that the Kruskal-Wallis is more powerful than the Friedman ANOVA when *= ,0, 

The data further indicate that nonparametric ANCOVA is more powerful than the 
Friedman ANOVA at all three levels of a when *= ,0, Consistent with asymptotic 
theory the data are not as clear when comparing nonparametric ANCOVA to the 
Kruskal-Wallis, Kruskal-Wallis is seen to be slightly more powerful at a * ,10, 
,596 compared to .543; and a « ,05, ,437 compared to .418; but slightly less 
powerful than nonparametric ANCOVA for a - ,01, .158 compared to .202, As o 

XY 

increased the power of the Kruskal-Wallis remained relatively stable whereas 
the powers of Friedman ANOVA and nonparametric ANCOVA were both strict mono- 
tonic increasing functions of for all three levels of c. When > ,4, 
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the power of nonparamet ric ANCOVA was greater than the power of Kruskal-Wallis 
and Friedman for all three levels of a. As noted In our 1970 paper, the data 
make no clear distinction on the basis of power between Kruskal-Wallis and 
Friedman ANOVA when correlation is .4* When the correlation is .6 or .8, 

Friedman was clearly more powerful than Kruskal-Wallis. There was a slight 
tendency for the nonpar ame trie ANCOVA to become progressively more powerful 
than the Friedman ANOVA as increased. When the empirical power of the 
Friedman ANOVA is subtracted from that of the nonparametric ANCOVA for in- 
creasing values of p , the differences for ot ■ .10 are .015, .013, .028, 

A I 

.064; for n B .05 are .086, .091, .162, .120; and for a « .01 are .007, .082, 
.112, ,168, The values of P^y were .0, .4, .6, and .8 respectively. 

Table 2 compares the empirical sampling distributions of the three para- 
metric tests when calculated on data in their respective appropriate designs 
for t - 3, b * 8 and varying values of p^y. As should be expected, the three 
rows labeled central indicate very good fit for all levels of a. The one 
exception is one-way ANOVA with « .6, which resulted in empirical a's .010 
too large for all three nominal values of a. 

The empirical pov'ers indicate that when p^y * .0, one-way ANOVA is more 
powerful than ANCOVA for all three levels of a. One way ANOVA ia also more 
powerful than two-way ANOVA on data in randomized block form for a ■ ,05, .458 
compared to .453; and .01, .210 compared to .199; but less powerful for a * .10, 
.584 compared to .598. As increased the power of one-way ANOVA remained 
relatively stable, whereas the powers of two way ANOVA and ANCOVA were both 
strict roonotonic increasing function of p^y for all three levels of a. Per- 
haps the most interesting power comparisons are between two way ANOVA and ANCOVA. 
The empirical powers are only partially in support of Cox's (1957) statements 
cited earlier in the paper. For p w « .0 the comparisons ate not of interest 

AI 



EMPIRICAL PROPORTIONS OF ONE AND TWO WAY ANOVA AND ANCOVA (PC) TEST 
STATISTICS FALLING IN THEIR RESPECTIVE REGIONS OF REJECTION FOR 
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since one-way ANOVA is preferred. When p^y = *4, two-way ANOVA vas more power- 
ful than ANCOVA when a = .10, ,645 compared to .625; but less powerful when 
ot » .05, .502 compared to .504; and a 3 .01, .223 compared to .239. As p X y increased 
from .4, ANCOVA increased in power at a faster rate than two vay ANOVA for all 
three levels of a with the differential rate of increase in power being most 
noticable for a ° .01. 

Taken together. Tables 1 and 2 afford a comparison of nonparametric 
techniques to their parametric analogues. As vas expected, the empirical a *s 
in Table 2 are in general slightly closer to their respective nominal values 
than are the empirical a's in Table 1. This finding reflects the discreteness 
of the nonparametric test statistics. As was suggested by the statements of 
asymptotic relative efficiency presented earlier, there was a greater loss in 
power going from the two way ANOVA to the Friedman than there was going from 
the one way ANOVA to the Kruskal-Wallis or from the parametric ANCOVA to the 
nonparametric ANCOVA. The earlier statements of asymptotic relative efficiency 
also suggested that as increased the nonparametric ANCOVA would become 
increasingly less powerful than the parametric ANCOVA. When the empirical power 
of nonparametric ANCOVA is subtracted from that of parametric ANCOVA for in- 
creasing values of p^y, the differences for a - .10 are .025, .031, .045, .043; 

for a ■ ,05 are .023, .037, .062, .063; and for a - .01 are^.004, .010, .041, 

.098. 

Table 3 provides average mean square errors and their standard nrrorr 
for nonparametric ANCOVA, one and two way ANOVA, and parametric ANCOVA for 
t - 3, b - 8 and varying value* of p . For all statistics but one way ANOVA 
the average mean square error decreased as increased which is reflected 
in the power trends reported for Tables 1 and 2. For the parametric statistics 
the values are identical for the central and noncentral cases. The expected 

0 

ERJC 



17 



Ul 

^4 



<< 

H 



^ 00 

o • 

CM 

i© 

£ ’ 

o * 

O <f 

5 : 

a o 




















t— l 


r- 


rH 


r- 










CO 


n* 


co 






CJ 


m 


o 


m 


o 






pH 


CO 


o 


co 

• 


o 

• 








m 


Cl 


m 


Cl 










-J 










O >N 


<r. 


o 


Oi 


o 




00 




CO 


o 


co 

• 


o 








m 


CM 


in 


CM 






<U >N 


O 


Cl 


o 


Cl 






f3 Cd 


r-> 


o ! 




o 






O 5* 


Oi 


o 


Oi 


o 

• 








00 


CM 


co 


CM 








o 


i£> 


o 


s© 






o 




O 


<* 


O 






PH 


iO 


o 


vO 


o 








CM 


vO 


-M 


vO 






o >> 


o> 




Ci 






\D 


s <d 


MO 


o 


iO 


O 






H 5* 


iO 


o 


MO 


O 










H 












Cl 




oi 








CD >s 


CO 


Oi 


CO 


Ci 


o 




c <d 


o 


O 


Oi 


o 


M 

ps 




o S 

. 


o> 


O 


Ci 


o 


g 






CM 


CM 


CM 


CM 


g 






LO 


00 


m 


00 






o 


CO 


O 


co 


o 


s 




pH 


00 


O 


00 


o 


CL, 




















oi 


Ci 1 


Cl 


Cl 






O >s 


o 


ci L 


1 o 


CI 






3 cd 


•n 


o 


in 


o 






H 5 


CO 


o 1 


00 


o 








CO 


m I 


CO 


in 






01 X 


vf 


Cl [ 


J 


d 






c id 


Cl 


o 


i d 


o 






O Dc 


Cl 


o 


1 Oi 


o 

• 








r-. 


CM 1 


\ N 










m 


O 


1 m 


CM 






u 


o 


rH 


1 o 


O 






Pu 


o 


O 

• 


H o 


rH 

o 








rH 




1 rH 


• 








Ci 


o 


y ci 








0 So 


CM 


CM j 


1 CM 


o 




o 


3 n3 


O 


H 


a o 


CM 






H ^ 


O 


o | 


H o 


rH 










• 




O 








rH 




H «H 


" 








00 


CO 


y oo 


00 






OJ 5o 


m 


cn 


1 wo 


oi 






c id 


o 


O 


i ° 


o 






ode 


o 


O 


n o 


o 










• 












rH 




Q ^ 










I H 


m 


co 


CO 








o 


CO 


'sO 


co 




oo 


o 


CM 


CM 


rH 


ci 






53 




CM 


r-. 


rH 








O 




r-. 










CM 




rH 










rH 


Cl 


o 


CM 


(_) 






CO 


o 


O 


CM 


►H 


\0 


O 


in 




CM 


m 


cc 


i 


53 


rH 


CM 


00 


CM 


K 






CO 




n* 




g 






CO 




CM 




2 






rH 




o 


O 


n. 






o 


Cl 


CM 


ro 


g 


sj 


u 


U 00 


co 


CM 


M0 


o 




53 


0 co 


CM 


O 


CM 


53 






• 

rH 




m 














ro 










■n 


'SJ 


il 


®r ,— 








Ci 


sO 


rH 










o 




c> 


rH 




o 


U 


W0 


hO 


o 


<r 






53 


00 


rH 


o 


CM 








• 


• 








i 








o 














<r 





TVH 1N3J I] TWmjKO K ^ 



w 

u 

cd 

o 

O* 

C*J 



§ 

& 

cd 

*4 

<y 

§ 

8 

RJ 

3 

O 

in 

*H 



w 

<U 



o 

CJ 

e 



c 

v 

a> 



18 



in this row are standard errors of the average mean squares 



-14- 



value of the mean square error for ANOVA is one for all values of and in 

each case the empirical values were very close to one. The expected value of 

2 

the mean square erior for parametric ANCOVA is (1 - p ) 20 w hen t = 3 and 

• ' 19 

b = 8, and again Ln each case the empirical values were in close agreement with 
the expected values. When p^y E the expected value of the mean square error 

for two way AN0 T A is one and tb-t-1 for ANCOVA. As p v increased from .0, the em- 

plrical average mean square errors for parametric ANC1VA decreased in size at 

a faster rate than those for two-way ANOVA. Although the average mean square 

error for parametric ANCOVA, .8352, was smaller than the average mean square 

error for cwo way ANOVA, .8509, when p^y « .4, there was no clear difference 

in their/ empirical powers as reported earlier in reference to Table 2. For non- 

parametric ANCOVA the noncentral average mean square error was smaller than the 

; 

central average mean square error for all values of p^y. This result is ex- 
plained by the nature of the dependent variable. For the central case the de- 
pendent variable ranks 1 through tb are evenly distributed across levels of 
the independent variable. When the noncentral case is analyzed, the larger ranks 
(when observations n re ranked from s. ’lest to largest) tend to be in the 
group whose raw data values on the dependent variable were all increased by 
one. It follows that the variance of the ranks in the group whose values were 
increased by one will be reduced and the within groups variance of ranks for 
the other levels of the independent variable also vilJ be reduced but to a 
lesser extent. 

Table 4 presents the average correlations of the antecedent and depen- 
dent variables within treatment groups, p^, and across all ranked observations, 

P S> for the populations investigated. o y is the same for both the central and 
noncentral cases and i9 in close agreement with its population value p^. Ex- 
^ cept for pj^ « .0 the central case values of pg have corresponding population 
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values less than p as indicated by the relationship given earlier between 



Pearson Product-Moment and Spearman correlation coefficients. The decrease in 
magnitude is .015 for p ■ .4, .018 for = .6, and .014 for p w = .8. The 
empirical values of pg were in general agreement with their corresponding param- 
eters. Except for p^y “ .0# the noncentral values of pg were smaller than their 
respective central values. There are two reasons for this finding. First, p g 
is the average of Spearman correlations calculated disregarding levels of the 
independent variable rather than the average of pooled within levels coefficients. 
Second, the noncentral rank data has the restriction placed on it that wa3 given 
earlier as an explanation of the difference between central and noncentral average 
mean square errors for nonparametrie ANCOVA. 

Table 5 compares the three nonparametrie test statistics for p * .6, b - 8 

Al 

and varying values of t. The central case empirical distributions again indicate 
general good fit for all three test statistics, with a slight tendency to be 
conservative at a * ,05 and .01 except for nonparametrie ANCOVA at t = 3 and 
a * ,01. Inspection of the three rows labeled noncentral indicates that all 
three statistics lost power as t increased which is understandable from the 
method used to define the noncentral case. Nonparametrie ANCOVA for all levels 
of a and Friedman AN0VA for a a ,05 tended to lose power at a slower rate than 
Kruskal-Wallis, When the empirical power of the Kruskal-Wallis is subtracted 
from that of the nonparametrie ANCOVA for increasing values of t, the differences 
for a - .10 are .121, .160, .204; for a « .05 are .155, .187, ,219; and for 
o» * .01 are .166, .180, .179. Kruskal-Wallis powers subtracted from Friedman 
AN0VA powers for increasing values of t give differences for a ** .10 of .093, 

.095, .049; for a « .05 of .025, .080, .118; and for a « .01 of .054, .031, 

.051. Nonparametrie ANCOVA and Friedman AN0VA tended to lose power at the 
same rate for increasing values of t with nonparametrie ANCOVA the more 
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power^ul for ill values of t and all levels of cx. The data also indicate that 
nonparametric ANCOVA and Friedman ANOVA are more powerful than Kruskal-Wallis 
when p * .6, b * 8 and all values of t. 

At 

Table 6 compares the three parametric tests for p^ = .6, b = 8 and 
varying values of t. Again as was to be expected the parametric tests indi- 
cated slightly better fit of empirical a*s to nominal oc f s with the previously 
noted exception of one way ANOVA when t » 3. The empirical power compa risons 
among the parametric test statistics in Table 6 are roughly parallel to those 
reported for the nonparametric tests in Table 5. All three parametric tests 
lost power for increasing values of t. There was a tendency for ANCOVA to 
lose power at a slightly slower rate than one way ANOVA at a =* .10 and .05 
for increasinf values of c. When the empirical power of one way ANOVA is 
subtracted from that of ANCOVA for increasing values of t, the differences 
for a « .10 are .149, .180, .192; for a « .05 are .163, .174, •181; and cor 
a =* ,01 are ,149, .151, ,132. However, in mild contrast to the results for 
nonparametric statistics the data do not suggest a differential rate of loss 
in power when comparing two way and one way ANOVA. ANCOVA and two way ANOVA 
tend to lose power at the same rate for increasing values of t. Contrary to 
the results for nonparametric statistics, there was no clear difference in 
power between ANCOVA and two way ANOVA when » ,6 # For t - 3 and 8 at all 
three levels of a, ANCOVA was slightly more powerful th£n two way ANOVA. The 
reverse was true for t = 5 at all three levels of a. For p * .6, b *= 8 at 

aI 

all three levels of t, ANCOVA and two way ANOVA on data in a randomized blocks 
design were clearly more powerful than one way ANOVA which is consistent with 
the nonparametric results. 

The loss in power suffered by using a nonparametric test when the as- 
sumptions for it 8 parametric analogue are met is seen by comparing Table 5 to 
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Table 6. The results are closely parallel to those reported earlier when com- 
paring the nonparametric powers In Table 1 to the parametric powers In Table 2. 
Again the loss was relatively small and quite similar for Kruskal-Wallis and 
nonparametric ANCOVA. For all values of t and all levels of a the Friedman 
ANOVA suffered the greatest loss when compared to its parametric analogue, 
which is consistent with asymptotic theory* However, the amour*: of loss did 
not decrease with increasing values of r as might have been expected from 
asymptotic theory* 

Table 7 presents nonparamet ric ANCOVA, one and two vay ANOVA, and ANCOVA 
average mean square errors and their standard errors for p « .6, b « 8* and 

Ai 

varying values of t. The parametric values were the same for both central and 
noncentral cases and the nonparametric ANCOVA values were systematically smaller 
in the noncentral case for the ? ame reasons given earlier when discussing the 
results in Table 3, For all levels of t the average mean square errors for one 
way ANOVA and ANCOVA are very close to their expected values. Consistent with 
theory, the average mean square errors for two way ANOVA are less than those 
for one way ANOVA but slightly larger than those for ANCOVA. The increasing 
values of the average mean square errors fnr nonparametric ANCOVA as t in- 
creased can be understood by remembering that the variance of ranks must in- 
crease ?>s the number of units being ranked increases. 

Table 8 compares the three nonparametric test statistics for p YV = *6, 
t * 3, and varying values of b. The three rows labeled central indicate good 
fit for the three statistics with the results indicating the Friedman ANOVA to 
be slightly liberal at b » 3 and a « .10 and the nonparametric ANCOVA to be 
slightly liberal at a » .01 for all values of b. The three rows labeled non- 
central indicate that all three tests increased in power as the values of b 
increased. The data suggest that nonparametric ANCOVA and Friedman ANOVA 
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increase in power at a faster rate than Kruskal-Wallis as val'ies of b increase. 
When the empirical power of the Kruskal-Wallis is subtracted from that of the 
nonpararoetric ANCOVA for increasing values of b, the differences for a - .10 
are .124, .121, .153; for a 3 .05 are .108, ,155, .179; and for a « .01 are 
.091, .166, .222. Kruskal-Wallis powers subtracted from Friedman ANOVA powers 
for increasing values of b give differences for a - .10 of .114, .093, .045; 
for a = ,05 of .013, .029, .063; and for a = .01 of ".027, .054, .090. The 
data indicate a slight tendency for nonparametric ANCOVA to increase in power 
at a rate faster than Friedman AN0VA for increasing values of b with nonparamet- 
ric ANCOVA the more powerful for all values of b and all levels of a. When 
the empirical power of the Friedman ANOVA is subtracted from that of the non- 
parametric ANCOVA for increasing values of b, the differences for a = . 10 are 
.010, ,028, .108; for a = ,05 are .095, .126, ,116; and for a ■ .01 are .118, 
,112, ,132, Again nonparametric ANCOVA and Friedman ANOVA are more powerful 
than the Kruskal-Wallis when * .6, t - 3, and varying values of b with 
the one exception of Friedman ANOVA when b a 5 and a. « .01. For all popu- 
lations represented in Table 8, the nonparametric ANCOVA was more powerful than 
the Friedman ANOVA. 

Table 9 compares the three parametric test statistics for p^y » .6, t « 3, 
and varying values of b. The empirical a’s presented in the rows labeled cen- 
tral indicate good fit with the same exception noted earlier of one way ANOVA 
when b & 8. Again the parametric results closely parallel the corresponding 
nonparametric results in Table 8. All three statistics Increased In power as 
the values of b Increased. Both ANCOVA and two way ANOVA Increased In power 
at a faster rate than one way ANOVA for Increasing values of b. When the empiri- 
cal power of the one way ANOVA is subtracted from that of ANCOVA for increasing 
values of b, the differences for a » .10 are ,143, ,149, .147; for a - .05 are 

o 
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.105, .163, ,216- and for a *» ,01 are .045, .149, .213. One way ANQVA powers 
subtracted from two way ANOVA powers for Increasing values of b give dif- 
ferences for a = .10 of .094, .125, .137; for a - .05 of .079, .124, .195; 
and for a = .01 of .043, .104, .190. ANC0VA and two way ANOVA on data in 
a randomized blocks design increased in power at about the same rate for in- 
creasing values of b with ANC0VA the more powerful for all values of b and 
all levels of a. For = *6, t = 3, and all values of b, ANC0VA and two 
way ANOVA were both more powerful than one way ANOVA. 

A comparison of the nonparametric empirical powers in Table 8 with the 
parametric empirical powers in Table 9 indicates the same relationships as 
similar earlier comparisons had shown for Tables 1 and 2 and Tables 5 and 6. 

Ihe difference in power is relatively small for Kruskal-Wallis versus one way 
ANOVA and nonparametric ANC0VA versus parametric ANC0VA, but quite substantial 
for Friedman versus two way ANOVA. 

Table 10 presents nonparametric ANC0VA, one and two way ANOVA, and ANC0VA 

average mean square errors and their standard errors for p * .6, t = 3, and 

XY 

varying values of b. The parametric values were the same for both central and 
noncentral cases and the nonparametric ANC0VA values were systematically smaller 
in the noncentral case for the same reasons given earlier when discussing the 
results in Tables 3 and 7. The increasing values of the average mean square 
errors for nonparametric ANTOVA as b increases are understood by the same 
reason give i for their having increased with increasing values of t in Table 7. 
Again for all levels of b, the average mean square errors for one way ANOVA and 
ANC0VA are very close to their expected values. The average mean square error 
for two way ANOVA decreased with an increase in the number of blocks as was 
suggested by Cox'y statement of true imprecision given earlier. 
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The purpose of the present paper was to extend understanding of the relative 
merits, when nonparametr ic statistics are used, of a simple random assignment 
design v ithout benefit of data on an antecedent concomitant variable, a simple 
random assignment design with information on an antecedent concomitant variable 
which is linearly related to the dependent variable, and randomized blocks de- 
sign. Interest also centered on the small sample goodness of fit properties 
of the Kruskal-Wallis, nonparametric ANCOVA, and Friedman ANOVA test statistics 
which are appropriate for the designs investigated. The data from our earlier 
study suggested that the randomized blocks design analyzed by Friedman's ANOVA 
is a successful method for improving power over that for a simple random assign- 
ment design analyzed by the Kruskal Wallis when the correlation between the 
blocking variable and the dependent variable is greater than .4. When the cor- , 
relation is equal to .4, power is not a relevant dimension for ctoosing between 
the two designs and when correlation is zero the simple random assignment design 
analyzed by the Kruskal-Wallis is the more powerful. Data from the present study 
indicate that Quade's recently proposed nonparametric ANCOVA on data in a simple 
random assignment design is more powerful than Friedman's ANOVA on data in a 
randomized blocks design for all values of the correlation between the concomi- 
tant variable and dependent variable. Further the nonparametric ANCOVA is 
equal in power to the Kruskal-Wallis when “ *0, but becomes progressively 

more powerful than the Kruskal-Wallis for increasing values of p . 

A I 

The data suggest similar statements for parametric statistics with a few 
notable exceptions. Two way ANOVA on data in a randomized blocks design was 
more powerful than one way ANOVA on data in simple random assignment design 
when the correlation between the blocking variable and the dependent variable 
was equal to or greater than .4. The data also indicate one way ANOVA as more 
nnwprful than ANCOVA for * .0. Further, the pow<- r of ANCOVA and two way 
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ANOVA were similar for = .4 and ANCOVA was more powerful than two-way ANOVA 
for p^Y 1 . When comparing the power of the three nonpa raise trie statistics 

to their parametric analogues where parametric assumption are met, one-way ANOVA 
and ANCOVA were only slightly more powerful than the Kruskal-Wallis and non- 
parametric ANCOVA respectively, but two way ANOVA was considerably more powerful 
than the Friedman ANOVA. 

The implications are that when information is available on an antecedent 
variable which has a monotonic relationship with the dependent variable and non- 
parametric statistics are to be employed, nonparametric ANCOVA is a better method 
for increasing power than is a randomized blocks design. However, a randomized 
blocks design may still be useful for improving power in experiments where the 
antecedent concomitant variable is measured on a nominal scale and nonparametric 
statistics are to be used. The data indicated that parametric ANCOVA became 
progressively more powerful than nonparametric ANCOVA as increased; it should 
be remembered that nonparametric ANCOVA needs only a monotonic relationship 
between the covariable and dependent variables to be effective. In situations 
where the rank order correlation exceeds the Pearson-Product moment, the non- 
parametric ANCOVA may well be more powerful than parametric ANCOVA. 

In general the data for the parametric tests were consistent with the 
literature. Our results indicate that simple random assignment analyzed by one 
way ANOVA is to be preferred when 0. For p B .4 there was no clear 

difference in power between ANCOVA and two-way ANOVA on data in randomized blocks 
design, but either is to be preferred over ANOVA. However, the literature has 
suggested that randomized blocks design would be more powerful than ANCOVA for 
°XY * The ^ ata alS ° 8u ^ est t ^ iat ^COVA is to b e Preferred over randomized 

blocks design analyzed by two way ANOVA for values of « .6 and that ANCOVA 
is clearly the more powerful for =* .8. 
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All three nonpar ametric and all three parametric methods increased in power 
with an increase in sample size, Nonparametric ANCOVA and Friedman ANOVA gained 
power at a faster rate than the Kruskal-Wallis , and parametric ANCOVA and two 
way ANOVA gained power at a faster rate than one way ANOVA. There was a slight 
tendency for nonparametric ANCOVA to gain power at a faster rate than the Friedman, 
but parametric ANCOVA and two way ANOVA gained power at the same rate. All three 
nonparametric and all three parametric methods lost power as the number of levels 
of the independent variable increased as was to be expected from the manner in 
which the noncentral case was defined. 

For three or more levels of the independent variable and five or more 
replications per level the empirical and nominal levels of a were in close agree- 
ment for all three nonparametric tests. The empirical u’s for the Kruskal-Wallis 
and Friedman tests tended to be slightly conservative except for t equal to three 
and b less than ten at a equal to .10. The empirical a’s for the nonparametric 
ANCOVA were slightly liberal for t equal to three and u equal to .01. For designs 
of smaller dimensions exact tests should be used. 
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